1. Field of Invention
The invention relates generally to digital image processing systems. More particularly, methods and apparatus for selectively enhancing a digital image are disclosed. More particularly, the invention provides light source color correction techniques suitable for use in both digital still and video image photography.
2. Description of Relevant Art
The color quality of light sources in general can be stated in terms of color temperature that defines the color of a light source relative to the visual appearance and is typically expressed in degrees Kelvin (xc2x0 K). More specifically, color temperature refers to the color of a blackbody radiator at a given absolute temperature. A blackbody radiator changes color as its temperature increases, first to red, then to orange, yellow, and finally bluish white at the highest temperature. Table 1 lists a variety of light sources and their corresponding color temperature in degrees Kelvin(xc2x0 K).
It should be noted that in reference to Table 1, sunlight is the light of the sun only whereas daylight is a combination of sunlight plus skylight. The values given are approximate because many factors affect color temperature such as the sun angle, and the conditions of the sky-clouds, haze, dust particles-raise or lower the color temperature. With indoor lighting, lamp age (and blackening), voltage, type of reflectors and diffusers affect tungsten bulbs; all of these can influence the actual color temperature of the light.
With conventional film photography, the color temperature of the ambient light source can, and usually does, has a significant effect on the finished product. By way of example, referring to Table 1, a photograph taken with standard daylight film in daylight (color temperature 8000xc2x0 K) will typically exhibit what is typically referred to as xe2x80x9cnaturalxe2x80x9d colors since the peak of the daylight spectrum is skewed to the blue end of the visible light spectrum. However, in the event that the same film is used to take a photograph in a room illuminated by a standard tungsten filament incandescent light source (color temperature approximately 2900-3000xc2x0 K), the photograph produced will more than likely have a yellowish cast. This yellowish cast is due to the fact that the light produced by a tungsten filament has a spectrum skewed to the yellow portion of the visible light spectrum. Conventional techniques well known to those skilled in the art of photography that attempt to compensate for the differences in ambient light color temperatures include the use of specially formulated film and/or the use of special filters that are physically attached to a camera.
Take, for example, the case where standard daylight film (i.e., film formulated for use in a color temperature range corresponding to daylight) is used to take a photograph in a room illuminated by a tungsten filament incandescent lamp. In this case, a special tungsten filter can be used to compensate for the difference in color temperature of the light produced by the tungsten lamp as compared to standard daylight for which the film was originally formulated. In this way, the photographic film is exposed to light more closely resembling standard daylight. In another case, when shooting a photograph in evening or night conditions under incandescent lighting (when a flash is not used), a yellowish cast can be readily seen. Again, a conventional attempt to correct this problems relies on the use of tungsten film or special filters.
Another approach commonly followed is to use specially formulated xe2x80x9ctungstenxe2x80x9d film that is specially formulated to absorb particular portions of the visible light spectrum of the light produced by the tungsten filament lamp. The disadvantage to this approach is that if this film is used outdoors, or in a room illuminated by, for example, a xenon lamp, then different color temperature correction is typically required. These particular approaches to color temperature compensation are potentially expensive and time consuming since additional equipment (such as filters) and/or film suitable for the particular ambient light conditions must be readily available to the photographer.
Although, either, or both, of these (and other) techniques have and can be used to color compensate for differing ambient light sources in conventional film photography, these techniques are not particularly useful with digital photography. Since digital cameras and the like use photoreceptors (CCD, CMOS, etc) as a form of digital film to capture the image in the form of photons, it is impossible to use xe2x80x9cspecialxe2x80x9d film to accommodate different ambient light conditions. Since a digital camera is limited to those photoreceptors originally included in the camera, the only realistic alternative is to use xe2x80x9con the lensxe2x80x9d filters to compensate for the color temperature of the ambient light source. Unfortunately, xe2x80x9con the lensxe2x80x9d filters for digital cameras can be expensive and difficult to use and as yet are not readily available. As with conventional film photography, since xe2x80x9con the lensxe2x80x9d filters absorb a particular range of color, the entire photograph is thereby altered even in those select portions of the photograph where color correction is not needed.
Recently, attempts at after the fact color correction have come to the fore. More particularly, IntellihancePro(trademark) manufactured by the Extensis Corporation of Portland, Oreg. and Adobe Photoshop(trademark) manufactured by the Adobe Corporation of Mountain View, Calif. provide algorithms that provide some rudimentary color correction. In the case of IntellihancePro(trademark), the removal of fluorescent lighting effects is very poor. Unfortunately, these algorithms can only be utilized on computing systems having substantial processing and memory capabilities (such as desktop computers) and as such can not be embedded in devices such as cameras, scanners, and the like. However, even with the use of substantial memory and processing, these products are still substantially manual in nature and provide little or no automatic color correction.
Also well known in the art, white balance is the adjustment of a camera so that it shows white parts of the picture without any color tint. However, in order to determine what is xe2x80x9cwhite,xe2x80x9d the electronics in the camera must be shown a white reference point under ambient light conditions. If this is done manually, then every single time light sources are changed, a new white balance must be determined, otherwise, the camera thinks white is what it saw under fluorescent lights when shooting in warm sunny summer sunlight, for example. Therefore, by adjusting the white balance on the camera it is ensured that colors are reproduced accurately since without proper white balance, camcorders, for example, would show the color as it truly is i.e. bluish in sunlight and reddish in normal room lighting.
Recently developed automatic white balance circuitry available in most digital cameras, such as camcorders, although an improvement over performing white balancing manually, sometimes simply does not give exactly the right result. For example, fluorescent lights can cause problems and fool the white balance system leaving a greenish tinge that can make people look ill. These problems then require xe2x80x9cafter takenxe2x80x9d processing to further xe2x80x9cclean upxe2x80x9d the image. Other problems can be caused when multiple cameras are being used to record the same event. If each camera has a slightly different color balance, then the resulting images will not color match.
Therefore, what is desired is an efficient method and apparatus for producing a color corrected digital image both in real time as well as for xe2x80x9cafter takenxe2x80x9d digital images that can be used in any suitable display device. Such display devices include TV displays, computer displays, still cameras (both digital and analog), digital video, and the like.
The invention relates to an improved method, apparatus and system for real-time enhancement of digital images. Broadly speaking, the invention relates to an improved method, apparatus and system for automatic real-time color temperature correction of digital images. In one implementation, the invention determines the particular ambient light conditions under which a captured digital image has been taken. Once a characteristic color temperature is ascertained, a set of correction factors are retrieved based upon the ambient color temperature. The captured digital image is then converted to a particular color space, which in a preferred embodiment is one of a number of hue-based color spaces. Such hue-based color spaces include, but are not limited to: Hue, Saturation, and Value (HSV), Hue, Saturation, and Intensity (HSI) and Hue, Lightness, Saturation (HLS), Hue, Saturation, and Brightness(HSB), Tek HVC (Hue, Value, Chroma). If the number of pixels in the digital image having its hue component within a specified range and whose saturation value is greater than a saturation threshold is greater then a pixel count threshold value, then the digital image is considered to exhibit an undesirable lighting effect and is color corrected. Otherwise, the original digital image is output to a display device.
In the case where the number of pixels in the digital image having both their respective hue component within the specified range and saturation value greater then a saturation threshold value, the digital image is color corrected. In this case, each pixel is color corrected or not based upon the particular color correction factors. The color-corrected image is then available for viewing on the display device.
The present invention is especially well suited for the removal of incandescent and fluorescent lighting effects from images. The present invention is also well suited for removing the yellowish-orange cast due to tungsten filament incandescent lighting as well as the blue/greenish cast due to fluorescent lighting.
These and other advantages of the present invention will become apparent upon reading the following detailed descriptions and studying the various figures of the drawings.